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Abstract — In this paper we consider the design of wire- 
less queueing network control policies with special focus on 
application-dependent service constraints. In particular we con- 
sider streaming traffic induced requirements such as avoiding 
buffer underflows, which significantly complicate the control 
problem compared to guaranteeing throughput optimality only. 
Since state-of-the-art approaches for enforcing minimum buffer 
constraints in broadcast networks are not suitable for application 
in general networks we argue for a cost function based approach, 
which combines throughput optimality with flexibility regarding 
service constraints. New theoretical stability results are presented 
and various candidate cost functions are investigated concerning 
their suitability for use in wireless networks with streaming 
media traffic. Furthermore we show how the cost function based 
approach can be used to aid wireless network design with 
respect to important system parameters. The performance is 
demonstrated using numerical simulations. 

I. Introduction 

Multimedia traffic, in particular streaming content, is in- 
creasingly gaining importance as it constitutes an ever larger 
fraction of the traffic observed in nowadays wireless networks. 
However, routing streaming content through a wireless net- 
work raises hard constraints on the network and corresponding 
control policies. 

Research on queueing control in stochastic networks 
long focused solely on throughput optimality (i.e. stabil- 
ity), following the celebrated MaxWeight policy of Tassiulas 
and Ephremides (TJ. However it was soon discovered that 
MaxWeight-type policies can lead to significant delays Q 
which may render their application in practice impossible. 
Significant research activities followed, focusing on delay 
reduction in backpressure based policies, such as |3]|4||5|). 
A general class of throughput optimal policies with improved 
delay performance is presented recently in [6|, and a general 
survey on recent policy synthesis techniques can be found for 
example in [7 |. However, delay clearly is not the only perfor- 
mance measure relevant for modern multimedia applications. 

In this paper we want to look at wireless network and 
policy design from a different angle. In particular we adopt 
a service centric or application centric view point. Our main 
motivation is that (especially with the ever growing amount 
of streaming traffic) it is of lesser importance to be e.g. delay 
optimal as long as the service requirements are fulfilled and- 
in case of streaming traffic-the stream is not interrupted. In 
fact, a throughput optimal or even more a delay optimal policy 
can be even harmful in case of size-limited buffers. In such 
a case it can be better to absorb traffic peaks at intermediate 
buffers instead of routing packets as fast as possible to the 
user. On the other hand, when the application is satisfied with 
the offered service it is not necessary to put huge efforts in 



small improvements of particular performance measures. By 
contrast, it is much more important that a network control 
policy is able to cope with changing services and thus with 
changing constraints, however without sacrificing stability. 

For control policy synthesis, we assume that the service 
layer constraints can be expressed as requirements on the 
buffer states. For example a general delay sensitive application 
may require certain maximum levels of the buffer states while 
a multimedia streaming application will additionally need 
minimum buffer constraints, such that the stream will not be 
interrupted due to buffer underflows. While the first type of 
constraint is well investigated, especially in multihop networks 
the second type bears serious challenges. In a way, this 
also implies a change of viewpoint: from stabilizing transmit 
buffers to controlling the user or application buffer^] 

While performance metrics such as throughput, delay (in- 
cluding deadline constrained traffic), and fairness are well 
investigated in the literature, the problem of minimum buffer 
constraints was to date mainly considered in the context of 
stochastic processing networks JS). Considering queueing net- 
works, previous works consider underflow constraints mainly 
in broadcast or simpler networks l9l fT0l . In 13 the problem 
of guaranteeing minimum buffer constraints in a network of 
multiple transmitter-receiver pairs with transmitters being able 
to cooperate is tackled in the following quite simple way. The 
available resources per user are divided into a fixed and a 
variable part. Essentially, whenever the user buffer states are 
low the transmitter increases the variable rate and decreases it 
when the user buffer states are high. However, this approach is 
limited to applications in broadcast or point to point links and 
cannot easily be generalized to the case of arbitrary networks, 
due to the complex interactions. For example, it is required 
that buffer levels for each user progress independently of other 
users ([9|). Clearly this is not the case in a multihop network. 
Already in the simple case of tandem networks, as depicted in 
Figure [TJ the approach is stretched to its limits when more than 
two buffers are involved, since all buffers are strongly coupled. 
Assume we want to steer the level of buffer m towards a 
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Fig. 1. Tandem Network 



certain value, in this case it is not anymore obvious how 
to control the output for example of the first buffer. This is 



Throughout the paper we use the terms 
buffer' synonymously. 



'user buffer' and 'application 
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even further complicated when different costs are assigned to 
different buffers. Apart from generalization issues to multihop 
networks, another drawback of these approaches is that they 
are rather static with respect to service constraints. 

The latter is more than a side note since in addition to guar- 
anteeing specific service requirements, a sophisticated control 
approach should also be flexible with respect to different and 
possibly changing requirements for different buffers in the 
network. In Q we introduced a control framework, called fi- 
MaxWeight, which-based on an underlying cost function-is 
highly suitable for application in such cases. Besides, when 
carefully designed we still maintain throughput optimality. 
In this paper we provide general sufficient conditions on 
the stability of a /i-MaxWeight based control policy and 
subsequently prove two simpler stability results which are 
based on additional assumptions on the network model. 

In summary, our approach guarantees stability of all buffers 
in the network and, being derived from an underlying cost 
function, allows incorporation of application requirements. 
Moreover it can flexibly react to changing user requirements 
by using different cost functions. 

As we will demonstrate, the choice of the underlying 
cost function is crucial for the performance of the resulting 
control policy. In particular we show that with a sophisticated 
cost function choice, buffer sizes of particular queues can 
be steered towards beneficial operating points. As a main 
performance measure to compare various cost functions we 
define a notion of queue outage to be minimized. This not 
only incorporates buffer overflows to capture effects of high 
packet holding costs or even dropped packets, but also buffer 
underflows to capture service interruptions. 

In addition to above mentioned benefits our framework can 
also be used to aid a priori design of important network 
parameters, in order to achieve a given required performance. 
Eventually-using numerical experiments-we attempt to an- 
swer the following two questions: 

1) Which cost-function is most suitable to minimize the 
probability of buffer outage events? 

2) How much system resources are needed to achieve a 
given required outage performance? 

As an example for the second issue we consider the problem 
of wireless bandwidth provisioning. 

The rest of the paper is organized as follows. In Section 
[II] we introduce our stochastic queueing network model and 
important stability definitions. In Section III we introduce 
important preliminaries and in Section |IV] we introduce our 
control approach and further present new theoretical results 
on the stability of resulting policies. Further we propose a 
pick-and-compare based variant, which significantly reduces 
complexity. Section [V] is dedicated to the choice of the 
underlying cost function and illustrates the implications using 
a simple tandem queue example. Using numerical simulations, 



Notation: We use boldface letters to denote vectors as 
well as matrices, and common letters with subscript are the 
elements, such that A, is the i'th element of vector A and By- 
is the element in row i and column j of matrix B. Moreover 
A T refers to the transpose of A. E{X} denotes the expected 
value of random variable X. Let I denote the identity matrix 
of appropriate dimension. Furthermore we denote 1 the vector 
of all ones. diag(ai,a2, ...) refers to a diagonal matrix built 
from the elements ai,<X2,"- and || • ||j denotes the U vector 
norm and ||x|| is an arbitrary norm. Furthermore we use A c 
to denote the complement of a set A. The probability of A is 
denoted as Pr{_4}. The indicator !{■} equals 1 if the argument 
is true and equals otherwise. 

II. System Model 

Similar to ifTTl we use a simple stochastic network model: 
the Controlled Random Walk (CRW) model. We consider a 
queueing network with m queues in total representing m phys- 
ical buffers with unlimited storage capacity. We arrange the 
queue backlog in the vector Q, such that Q = [Q\, . . . , Q m ] T 
which we refer to as the queue state. Let Ai be the set of queue 
indices. Suppose that the evolution of the queueing system is 
time slotted with t £ No. Then, the CRW model is defined by 
queueing law: 

Q (t + 1) = [Q (t) + B (t + 1) U (<)]+ + A (t + 1) (1) 

where [x] + = max{0,a;}. Here, the vector process A (t) G 
N™ is the (exogenous) influx to the queueing system with 
mean a G M™ (vector of arrival rates in packets per slot); 
B (i) G Z™ xi is a matrix process with average B G Z™ xi , 
containing both information about network topology (that is, 
connectivity or routing paths) and service rates. The control 
u = U (t) in slot t is an element of the set {0, 1}' constrained 
by Cu < 1 using the binary constituency matrix C G Zg mX ' 
(with l m > being the number of resource constraints in the 
network). For the sake of notational simplicity we omit the 
time index in the following where possible. Throughout the 
entire paper x G W n denotes the current backlog. 

In what follows, the queueing system ([T]i is assumed to be 
a <5o-irreducible Markov chain (<5o being the point measure at 
x = 0). 

A. Stability 

The stability of an <5o -irreducible Markov chain can be 
defined in different manners. Throughout this paper we use 
the following definition of stability. 

Definition 1. A Markov chain is called f-stable, if there is 
an unbounded function f : K" 1 — > K + , such that for any 
< B < +oo the set B := {x : / (x) < B} is compact, and 
furthermore it holds 



limsupE{/(Q(t))} < 

t— >+oo 



-OO. 



(2) 



Section VI evaluates the performance of the control approach 
and exemplarily demonstrates how it can be used for the design 



of wireless networks. We conclude the paper in Section VII 



In the definition the function / is unbounded in all positive 
directions so that / (x) — > oo if ||x|| — ► oo. Choosing directly 
/ (x) = ||x||, Definition [T| is equivalent to the definition of 
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strongly stable [12] which implies weak stability. Clearly, for 
any / (x) which grows faster than ||x||, inequality (|2| implies 
that the Markov chain is strongly stable. We call a vector 
of arrival rates a. £ stabilizable when the corresponding 
queueing system driven by some specific scheduling policy is 
positive recurrent. 

A scheduling policy is now called throughput optimal if it 
keeps the Markov chain positive recurrent for any vector of 
arrival rates a for which a stabilizing policy exists. 

III. Preliminaries 
Let us introduce a cost function 



c(x) , 



assigning any queue state a non-negative number. Typically, 
the goal is to minimize the average cost over a given finite 
or infinite time period or some discounted cost criterion. The 
optimal solution to the resulting problems -which in discrete 
time can be modeled as a Markov Decision Problem- can be 
found by dynamic programming, which is, however, infeasible 
for large networks. A simple approach to queueing network 
control is the myopic or greedy policy. Such a policy selects 
the control decision that minimizes the expected cost only for 
the next time slot. 

In ifTTl . a cost function based policy design framework 
called /i-MaxWeight is introduced which is a generalization 
of the MaxWeight policy. Meyn considers a slightly different 
definition of the CRW model, which is characterized by 
queueing law: 

Q (t + 1) = Q (t) + B (t + 1) U (t) + A {t + 1) (3) 

The control U (t) £ N l is an element of the region 

U* (x) :=W(x)n{0, 1}', 

with 

U (x) := {u € R l + : Cu < 1, [Bu + a] t > for x t = 0} . 

In the /i-MaxWeight based control policy, the control vector 
is derived according to 



argmin < V/i(x), Bu + a. > 

u€U*(x) 



(4) 



Thus, the policy is myopic with respect to the gradient of some 
perturbation h of the underlying cost function. Meyn develops 
two main constraints on the function h: the first requires the 
partial derivative of h to vanish when queues become empty: 

dh 

— (aj) = ifx t =0 (5) 

Moreover the dynamic programming inequality has to hold for 
the function h: 

min < V/i(x), Bu + a >< —c(x) 

When h is non-quadratic, the derivative condition (|5]l is 
not always fulfilled. Therefore a perturbation technique is 
used where h(x) — ho(x), hence it is a perturbation of a 



function h . Two perturbations are proposed: an exponential 
perturbation with 9 > 1 given by 

"TT - 1 

and a logarithmic perturbation with 9 > defined as 

x 4 := Xi log (i + y ) ■ ( 6 ) 

While the first approach shows better performance in sim- 
ulations, the stability of the resulting policy depends on 
the parameter 9 being sufficiently large (determined by the 
considered network setting). This is overcome by the second 
perturbation which is stabilizing for each feasible 9, however 
it comes with the additional constraint 

dh , 



dx. 



(x) > exi, Vi G M, 



which is a significant limitation on the space of functions that 
can be chosen as Iiq. 

IV. /i-MaxWeight 

In this section, we give generalized sufficient conditions 
for throughput optimality of the systems ([T]l and d3). In what 
follows, we consider scheduling policies of the form 



u* (x) = arg min (fi (x) , Bu + a) 



(7) 



:Cu<l 



where fi (x) is a vector valued function M.™ M"\ which 
is called the weight vector for some actual queue state x. 
Note that fi is reminiscent of a vector field and can thus 
be interpreted as a scheduling field for which we present a 
stability characterization. Observe that by construction of the 
policy we can without loss of generality normalize the weight 
vector as 



(8) 



I/* Wlli 

and hence ||£(Wlli = 1. Furthermore, we assume that the 
resulting policy is non-idling, i.e. ||/iWlli — if and only if 
x = 0. 

Theorem 1. Consider the queueing system (J7J driven by the 
control policy Q with some scheduling field fi. The policy is 
throughput optimal if the corresponding normalized schedul- 
ing field given in Eqn. ([S]) fulfills the following conditions: 

1) Given any < e\ < 1 and C\ > 0, there is some 
Bi > so that for any Ax eR m with ||Ax|| <Ci, we 
have \ fii (x + Ax) — /2j (x)| < t\ for any x £ with 
||x|| > B x , yieM. 

2) Given any < 62 < 1 ond C2 > 0, there is some 
B2 > so that for any x £ ]R™ with ||x|| > B2 and 
Xi < C% we have /Si(x) < 62, for any i G A4. 

Moreover, for any stabilizable arrival process the queueing 
system is f-stable under the given policy where f is an 
unbounded function as defined in Definition [7] The exact 
formulation of f depends on the field ju(x)- 

Proof: The proof is based on [5| and is provided in full 
detail in 1T3). ■ 
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With further assumptions on the underlying network these 
conditions can be significantly simplified. The following ad- 
ditional requirement on the network topology (also assumed 
in Theorem 1.1 of ifTTIO is needed for subsequent corollaries. 
• Bij(t) > —1 for each i,j and t, and for each j E 
{1, . . . , l u } there exists a unique value ij E {1, . . . , 1} 
satisfying 

Bij(t)>0 a.s. Vi^ij. (9) 

Corollary 1. Consider the queueing system driven by the 
control policy Q with some cost function h. Suppose the 
corresponding scheduling field fi(x) := Vh(x) is continu- 
ously differentiable and Condition ([£]) on network topology 
{£?(•)} holds. Then, the following conditions are sufficient for 
throughput optimality: 

1) For any e > there is some C* > so that for all 
Nl > C{: 

||Vlog(/Xi (as))|| < e, Vt e X 

2) If Xi = then fa{x) = 0, Vi e AL 

Proof: By Condition 2) of Corollary [T] we can assume 
that the random walk evolves on IR" 1 . Hence, we can skip 
Condition 2) of Theorem [T] since this condition (as its coun- 
terpart in Corollary [TJ ensures positivity of the random walk. 
We need to show that from 



|| V log ^ (x) || <e, ViEM, \\x\\>C 6 (e) 
(where C§ (e) is sufficiently large) it follows: 
fa(x + Ax) fa(x) 



EM Mj'W 



< e 



(10) 



(11) 



For orientation, let us assume more restrictive conditions 
first: take fa, Vi E A4 Lipschitz continuous and let 
J2jeM Mj( a; ) — > oo if ||x|| — > oo. Note, that these conditions 
akeady encompasses Meyn's perturbation (|6]l together with 
e.g. a linear cost function. 

It is easy to prove the corollary with these assumptions: by 
the mean value theorem we have 

fa (x + Ax) = fa (x) + V^fa (x) Ax 

where x is an (arbitrary) point on line connecting x and 
x+Ax whereas a; is a point connecting x+Ax. Since the field 
is Lipschitz we have Vl^faix) < Cy uniformly. Furthermore, 
since the policy is non-idling YljeM ( x + Ax ) — ^8 
where the normalization constant Cg can be chosen as large 
as possible without altering the policy (by the construction of 
the policy). Moreover, since J^jeM ^j( x ) ~* °°> \\ x \\ ~> 00 ' 
condition ( |TT) is equivalent to 



\fa (x + Ax) - fa (x)\ <e 



M ( x ) 



and, again, by the mean value theorem: 

| V T fa (x) Ax | < e faj (x) 

jeM 



Here, we tacitly assumed that we have selected x accordingly. 
Since Ace is fixed and by the positivity of fa it is sufficient 
that 



\\Vfa (x)\\ < 



\Ax\ 



-fa (x) 



which is equivalent to condition ( lOl with some ||x|| > Cq (e') 
(e' slightly smaller). 



Let us now prove the general case. Condition (lOi can be 
written as 

\7 T fa{x)Ax = e n , 



^2 jeM ^'(^O 



for some x with ||x|| > C (e„) where e„ is a zero sequence 
and C (e n ) is strictly increasing for any fixed Aa; E W n . Now, 
again, by the mean value theorem 



fa(x + Ax) 



fa{x) 



T,j £ M fa( x ) + ^ T fa{x)Ax E je ^( fai x ) ~ V T fa{x)Ax 

(12) 

where we set x as before and let x + Ax = x and x + 
Ax = x + Ax. x, x are points on the line connecting x and 
x respectively x and x + Ax. Note that faj{x) is zero if and 
only if fa(x + Ax) and fa(x) are both zero since otherwise 



< e, 



by condition ( 10 1 the gradient would be zero as well. Since in 
this case the condition is trivially satisfied so that we exclude 
it. 

Hence from ( fT2) i it follows 



(A) 



E ? -eA^Mj(^)(l 



fa(x + Ax) - fa(x)- 



\7 T fa(x)Ax 

faj(x) 



V 1 fa,(x)Ax 

faj(x) 



IB) 



< e 



jeM 



1 



V T fa{x)Ax 

fa(x) 



We can prove that, because of condition (lOi, (A) and (B) are 
zero sequences: suppose V T fij(x) is non-zero (then we can 
stop anyway) then by the repeated application of the mean 
value theorem, the denominator of, say, (A) can be written as: 

fa(x) = faj(x) + V flj(x 2 )Ax2 

This process generates sequences in K™ with x = x\,X2, ■■■ 
and Ax = Axi C Ax 2 , ... which are bounded and hence 
we can pick subsequences converging to some set of limit 
points x<x) , k = 1,2, .... Note that we can restrict the number 
of limit points to at most two since by definition every limit 
point is visited arbitrarily often and infinitely close and by 
construction of the sequence there is no possibility of more 
than two limit points which neither contain the other in 
between them. Take these two limit points with corresponding 
subsequence x£\ k — 1, 2: by continuous differentiability we 
have fa(x^) -> fa(x^) and Vfa,(xn^) -> VHj(xoo), k = 
1,2. It must also hold in the limit: 

fa(x£) + V T fa(x£)(x£ - = fa{x%) 
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(and vice versa). Since then 

V fJ,j [Xqo )(Xoo ^oo ) 
f^ji^oo )/^j(«^oo ) 



< e, 



(and vice versa) where e > is arbitrarily small by condition 
(10 1 we conclude that p,j(xo$) — Hj(x^)) (but not necessar- 



ily Xoo — ^oo )• 

Now, we can proceed the process sufficiently often as 



fj,j(x) 



V T ^Lj(Xi)/S.Xi 



< ^(^)(l+ VT ^ ( g^ 2 ) 

< ■■■ 

such that in the final step 

V T ^(a; n+ i)Aa;„ + i _ (V T /z j (a;^ ) ) + e k n )Ax n+1 



p,j(x ( ^)+e l n 
<e, fc,/ = l,2, 



by condition ( 10 1. Hence, we have 



e^,e" zero sequences, and further 



= 1 + e'" 



| Mi (x + Ax)- M i(x)(l + e^)| 

< |^(a; + Aaj) - m(x)\ + (H(x)e% 

which is equivalent to: 

\lH(x + Ax) - fj,i(x)\ < e + <4) " C^( x ) 



Since cc is arbitrary and can be suitably choose, condition ( 10 1 
with some ||as|| > Cq (e"") is sufficient for the latter to hold. 

■ 

Corollary 2. Suppose, everything is as in Corollary 1. Let 
the scheduling field be defined as (J,(x) := Vh (x) for some 
given simple perturbation x. Then, for some e > 0, 

dx i dx~i 

— — is Lipschitz, and — > oo,Xi — > oo, 

axi ' oxi 



dh dh ( d£i 

— — is Lipschitz, and — — \x) > — — 
axi oxi V oxi 



; Xi 



OO, 



Proof: We can write 
dh 



Hi(x) 



where we defined I 



(x) = l{Xi)^-(x) 

oxi axi 



dx) 



-g^. Note, that here Xi only depends 
on Xi. The gradient of the weight fii(x) is given by: 

dl 



djM, , _ 

dxi [X> )^^(x)-l( Xi ) 



^- {xi )^{x) + l{ Xi )^{x) i = j 

Define x A := x + Ax and x A := x(x A ). From the proof of 
Corollary [T] it is clear that we only have to show that 

|V T /i i (a;)Aa;| 



< e, 



for some e > arbitrarily small. This can be rewritten as: 



+ 



A\ dh i - A 



^2jeM,j^i dx j dx° (^AXj 

£ieV(*f)§*(* A ) 



< e 



Since , Z are Lipschitz, thus ^r- , ^ are uniformly 

oxi ' 1 oxj oxi 1 axi J 

bounded, and l(xi), ^^(x) > l 1+e {xi) — > oo when Xi — > oo, 
the effect of Ax vanishes in the denominator. The condition 
fl fl "(^) > l 1+e (xi) is required since we have expressions of 
the form 

l(Xi)l{Xj) 



which then become arbitrarily small. ■ 

A. Handling Complexity: A Pick and Compare Approach 

Centralized throughput optimal scheduling policies, such 
as MaxWeight, usually suffer from a high computational 
complexity. This naturally also applies to a /i-MaxWeight 
based policy. More precisely, a large computational burden 
arises since the selection of the best control vector in |7]) 
has to be carried out in every time slot, since the number of 
candidate control vectors grows exponentially with the size of 
the vector. To tackle the complexity issue there are several 
known approaches, such as randomized pick-and-compare 
based methods [14| which reduce complexity at the expense 
of higher delay or Greedy/Maximal scheduling [ 15] which has 
good delay performance but achieves only a fraction of the 
throughput region. 

To circumvent the complexity problem we propose a ran- 
domized version based on the first approach. As noted for 
example in lfl4l and [16|, throughput optimality can be 
preserved by using a linear-complexity randomized Pick-and- 
Compare method. Tailored to /i-MaxWeight, the approach can 
be summarized as follows: At t = 0, use u(0) = u E U* 
chosen randomly. Afterwards in each timeslot t > first pick 
a control u E U* randomly, and second, choose the control 
u(t) of this particular timeslot to be either the vector it if 



is sufficient for stability. 



(fJ>(x), Bu + a) < (fi(x), Bu(t — 1) + a) . 
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or u(t — 1) otherwise. Above algorithm preserves throughput 
optimality as long as 

F(u = u*) > S, 

for S > 1 14 1 [16 1 (which is trivially satisfied). The reduced 
complexity, however, comes at the expense of a higher con- 
vergence time. Yet, a tradeoff can be achieved by repeatedly 
applying the pick and compare steps in every particular 



timeslot. In the simulations presented in Section VI we apply 
the procedure n s times per timeslot. 

Note that, although not focus of this paper, the randomized 
algorithm was shown to be amenable as a basis for implement- 
ing decentralized throughout optimal control policies [17|[18|. 

V. Cost-Function Choice 

A vital design choice in the proposed control approach 
is the underlying cost-function to be minimized. Different 
applications induce different constraints on network control 
and thus need different cost functions. As mentioned before, 
applications may require both minimum and maximum buffer 
constraints. Assume we want to find a cost function that is best 
suited to steer the buffer levels of application buffers towards 
a target buffer state Q. 

In previous work such as ll5l lfT9l a linear cost function was 
used, given by 



c(Q) 



(13) 



since the aim was to minimize total buffer occupancy, corre- 
sponding to end-to-end delay. However, this cost function is 
unsuitable to avoid buffer underflows since it does not penalize 
buffer states below the target level. A simple and straight 
forward cost function choice that penalizes deviations from 
target buffer state Q in both directions is the shifted quadratic 
cost function, given by 



(14) 



However ( 14 1 naively treats all buffers in the network equally 
although most likely only application buffers have minimum 
state constraints. 

In fact, a better performance can be observed by combining 



(|T3jH 14 1, such that only user buffers have quadratic cost 
terms, while all other buffers induce linear costs. We call the 
resulting function composite cost function, given by 



c(Q) 



(15) 



where Q denotes the desired target buffer level of the applica- 
tion buffers^] and l u denotes the set of all user buffer indices. 



Of course, the shifted quadratic terms in (15 1 are only one 



of many possibilities to steer the buffers towards a desired 
working point. Intuitively, any cost function that produces low 

2 For simplicity we assume all application buffers have the same target 
buffer level. 



costs around the target level and increasing costs for under- 
flows and overflows should lead to the anticipated behaviour. 
Another approach is to explicitly design a cost function with 
desired properties. Motivated by the required behaviour we 
additionally consider a cost function which is inspired by the 
transfer function of a Butterworth bandstop filter, which we 
subsequently call bandstop cost function. 
The resulting cost function, is given by 

c(Q) = £ c ™* 1 / . Nfc + E c i^i- (16) 

\ i + j 

Thereby the parameter k is the analogue to the filter's order, w 

s Cost Functions (Single Buffer) 




Bandstop CF [k=lD, c = I0e5) 
Bandstop CF |kz2, c = 10eS) 
Linear CF 

Shifted Quadratic CF 



BUFFER SIZE 
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Fig. 2. Cost function trajectories for a single buffer, 



10 b 



determines the width of the interval of buffer state around the 
target buffer level that produce very low costs, and c max scales 
the function to the desired maximum cost value that occurs 
when the deviation from the target buffer state is significantly 
large. For a single buffer system the considered cost functions 
are illustrated in Figure [2] 

Having chosen an appropriate cost function the question 
remains, how to construct a corresponding weight function fi. 
In order to guarantee stability, it is sufficient to show that the 
weight function fulfills the stability conditions of Theorem [T] 
For this purpose we employ a perturbation technique (cf. [ 1 1 1). 
In we pointed out that a simple way to construct a weight 
function is fj.(x) = P# (x) S7ho(x), where a perturbation 
matrix Pg (x) := diag ^1 — exp ^— — F~y)) ' s use( l- 
It is based on the following perturbation of variables: 



exp 



0(i + E^-) 



(17) 



It can be easily verified that the conditions given in Section 
IV hold for suitable ho, i.e. it is throughput optimal for any 
9 > ([5 1 [13 1). Here we directly use our cost function as 
ho- For sufficiently large buffers the stability conditions are 
clearly fulfilled (while the behaviour when buffers are close 
to zero is controlled by perturbation (fl7]i). 

In general for throughput optimality only the behaviour 
matters when buffers grow very large [ 6 1 . Therefore we are not 
necessarily confined to everywhere differentiable cost func- 
tions. It is also possible to use a for example piecewise linear 
function to construct the weight function and still maintain 
throughput optimality, by using an arbitrary subgradient in 
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(j7]i at non-differentiable points. Although the control that is 
chosen at these point may not necessarily lead to a negative 
drift, concerning the stability conditions given in Section IV it 
is sufficient to assume that the cost function is differentiable 
when || a; || is larger than some arbitrary B. 

A. Example: Controlling a network of queues in tandem 

Let us clarify the influence of the cost-function choice by 
considering a very simple network known as tandem queue 
(cf. Figure [TJ, comprising a number of m buffers in series. 
To model for example a streaming service, we assume traffic 
arrives at the first buffer with mean rate a and some application 
removes traffic from the m'th buffer at a constant rate R a . 
The output of buffers 1 through m — 1 can be regulated by 
the control policy. While 'ordinary' tandem queue networks 
(without considering a specific application) are investigated 
thoroughly in the literature (see e.g. Q), we have two addi- 
tional issues here. First, we have no explicit control over the 
rate at which data is extracted from application buffer Q m . 
Second, in addition to stability (i.e. boundedness of buffers 
from above) we also have minimum buffer state requirements. 

Consider now the most simple network of m = 2. The 
second queue is thereby considered as the application queue, 
thus the only control dimension remaining is whether to send 
traffic from queue 1 to queue 2 or not. How the buffer 
state can be steered towards the target buffer level using an 
appropriate cost function is shown in Figure [3] It depicts 
queueing trajectorie^] of this system for 20000 time slots 
using two different underlying cost functions. The dashed lines 
represent policies based on the simple linear cost function, 
given in ( [13) . This cost function obviously does not stop the 
second buffer from growing, since both buffers are weighted 
equally. By contrast, the solid lines are obtained using the 



composite cost function ( 15 I, which produces a quadratic cost 
at the second buffer when the buffer state deviates from the 
target buffer level. It turns out that this function stops buffer 1 
from sending further traffic to buffer 2 when the latter reaches 
a certain level. Instead the excessive traffic is queued at buffer 
1, since it generates lower costs. 

a Tandem Queue Trajectories 
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Fig. 3. Queue Trajectories under different cost-functions 



To increase readability we performed an exponential averaging of the 
(strongly fluctuating) buffer values over a time window of 100 time slots. 



VI. Design and control of large multimedia 
networks: numerical performance results 

Using numerical experiments we subsequently evaluate the 
previously introduced cost functions in a larger network de- 
signed for entertainment purposes, thus especially relying on 
streaming traffic. Figure ^schematically depicts the considered 
network. 



i 1 1 1 1 1 



/ 



\ 



Fig. 4. Schematic representation of considered multimedia network 

We have several wireless access points each serving a cer- 
tain number of user terminals, potentially running streaming- 
based applications, in its area. Some terminals which are 
located in between several access points can potentially be 
served by more than one access point. The access points 
are connected via a wired backbone network to a central 
application server. The server itself may in turn be connected 
to the Internet, thus traffic for the each user arrives in a random 
fashion. Each component in the system has a number of queues 
or buffers with different requirements. The considered system 
can for example be used to model a wireless entertainment 
system (for example inside an aircraft cabin l20l ). 

To account for the anticipated multimedia applications we 
define a notion of queue outage as a measure of a policy's 
performance (in addition to the average cost incurred by 
the application of a particular control policy). Therefore we 
first define a buffer underflow as the event when the size 
of a buffer falls below a predefined value The buffer 

underflow frequency F™ n (T) of user i is consequently defined 
as the sum of the timeslots t < T in which its buffer is 
lower than QW. Similarly we can define a buffer overflow 
event as an event where the buffer grows beyond a certain 
value Q( 2 \ The buffer overflow frequency i 7 ™ ax (T) of user i 
measures the number of timeslots in which its buffer is larger 
than Q^ 2 \ The total sum of buffer outages is consequently 
F oat {T) = T Ji eX u F f !i ( T ) + Fr x ( T )> and eventually the 
relative frequency of queue outage events is defined as 

P° ut (T) = J-^F^T), (18) 

where / denotes the number of user buffers. The values of 
QW an d Q(2) 

can be flexibly adopted to the requirements 
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of the desired application. The goal of the control policy is 
consequently to keep the buffer states in between and 
Q( 2 h Thus it is reasonable to choose Q = \ (QW + Q (2) ) 
for the target buffer state. 

Using this definition we want to evaluate the cost functions 
from Section [V] with respect to two main aspects. First we 
want to ask how the queue outage performance evolves with 
respect to varying traffic intensities. This gives us an estimate 
of the robustness of the various cost functions. Second we 
demonstrate how the proposed approach can be used as a tool 
for wireless network design. We clarify this by an example, 
evaluating how the performance evolves with varying system 
bandwidths. This allows an a priori assessment of how much 
bandwidth has to be provided in order to support service 
constrains expressed by a predefined queue outage probability. 

Traditionally the CRW network model summarized in Sec- 
tion [II] comprises static links. However in our system we 
assume time-varying wireless links between access points and 
terminals. Since we are mainly concerned with MAC-layer 
performance we use a simple abstraction of the wireless link 
capacities. For this, we apply a result from [21 1 which deter- 
mines the mutual information distribution of a multi-antenna 
OFDM-based wireless system. To obtain rate expressions we 
use the notion of outage capacity (also derived in [21 1) I 0u t,p o 
for a given outage probability p a , defined as the maximum rate 
guaranteed to be supported for 100(1 — p a )% of the channel 
realizations and is given by 



L -out,q 



EfZ< 



OFDM 



\/Var (Xofdm)Q 



with Q(-) being the Gaussian Q-function and p the desired 
outage probability. E[2ofdm] ar >d VoK-^ofdm) me determined 
according to above mentioned mutual information distribution. 

In the following we show performance results in an example 
network structured according to Figure |4] Configuration details 
of the simulations are summarized in Table [I] 

TABLE I 

General Simulation Parameters 



Parameter 


Value 


Simulation Duration (T) 

Number of Users per AP 

Number of Access Points 

Wired Link Capacity (R B ) 

Wireless Link Outage Probability (p ) 

Target User Buffer Size (Q) 

Minimum User-Buffer Constraint (Q^ 1 ') 

Maximum User-Buffer Constraint (Q( 2 )) 

Application Rate (R a ) 

Iterations of Pick-And-Compare (n B ) 


100000 time slots 
10 

3 

100 MbitVs 
0.01 

20 Mbit 
10 Mbit 
30 Mbit 
3 Mbit/s 
100 



A. Traffic Intensity 

Especially when traffic is generated outside of the con- 
sidered network it is reasonable to assume that the offered 
arrival rate can deviate from the anticipated operating point. 
Therefore we subsequently investigate how the candidate cost 
functions influence the policy's behaviour with varying traffic 
intensity. Note, that the application drains application queues 



at a rate of 3Mbit/s, thus intuitively one can expect that 
at traffic lower than this value the influence of underflows 
dominates while at traffic rates larger then this value overflows 
are more likely to occur. We first consider the buffer underflow 
probability, since as opposed to other state-of-the art control 
approaches, a main incentive considered here is to prevent 
service interruptions due to low buffers. Consider Figure [5] 
comparing the underflow probability of various cost functions 
together with classical MaxWeight as baseline. When the 



Relative Frequency of Buffer Underflows 



Linear CF 
-MaxWeight 
-Shifted Quadratic CF 
-Bandstop CF 
-Composite CF 




2.9 2.95 
Mean Arr. Rate (Mbit/s) 



Fig. 5. Buffer underflow frequencies obtained by different cost functions 

mean arrival rate equals the application service rate R a (or 
is larger) all policies produce low underflow frequencies, 
however one can already observe a performance gain from the 
more sophisticated cost functions (CF). The gain significantly 
grows when the arrival rates are slightly lower than R a . 



While MaxWeight and the linear CF (13i show almost the 
same high underflow frequency (since both controls do not 
penalize low buffer states at the application buffers), already 



the shifted quadratic CF (14i shows some improvement. The 
best performance is obtained with the sophisticated bandstop 
and composite CFs (given by (16i and ( |T~5] >, respectively) 
which assign different costs to application buffers and non- 
application buffers. However, the performance gap between the 
two is relatively small. Therefore we choose the much simpler 



CF ( 15 i in the following. However not only buffer underflows 
have to be avoided, in some cases it may be desirable to 
avoid large queues as well. Therefore we next investigate 
the performance with respect to the queue outage probability 
defined in ( 18 I, Figure [6] summarizes the results. Naturally, at 



Relative Probability of Que 



, i -MaxWeight. Linear Cost Function 
.i MaxWeight. Quadratic Cost Function 
MaxWeight 

.i MaxWeight. Composite Cost Function 




Mean Pck. Art. Rate 



Fig. 6. Relative queue outage frequencies of various cost functions 

traffic arrival rates lower than the application's service rate, 
all cost functions produce decreasing outages with increasing 
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arrival rates, since queue underflows become less frequent. 
Beyond the applications service rate, the policies without a 
sophisticated cost function rapidly increase the queue outages 
with further increasing traffic due to overflows. Only the 
composite cost function can further decrease the number of 
queue outages, since exceeding traffic is stored at buffers that 



produce lower costs (cf. the tandem network in Section V-A i 



B. Network Design 

Given a concrete application which requires a certain out- 
age performance, it is of great interest how many system 
resources have to be provided. For example when planning 
some entertainment network it is important to know how much 
wireless bandwidth is actually needed to support the desired 
services. Therefore we next investigate the influence of the 
system bandwidth, when a certain target outage rate is to be 



achieved. For this, we use the composite cost function (15 1, 
and again plot MaxWeight as a baseline. Figure |7]compares the 
relative frequency of user buffer outages for varying wireless 
bandwidths. Assume we want a queue outage probability 



(defined according to ( 18 1) of 2% not to be exceeded, marked 
by the dashed black line in Figure [7] Obviously a simple 
scheduling strategy which is not designed based on a cost- 
function such as MaxWeight is not able to push the outage 
performance below this limit. By contrast using the cost- 
function based control a wireless bandwidth of 20 MHz would 
be sufficient to guarantee the desired performance in the 
considered configuration. 



Relative Frequency of Queue Outage 




Available Bandwidth [Hz] 



Fig. 7. Comparison of policies with respect to system bandwidth 

VII. Conclusions 

We presented a framework for the design and control of 
queueing networks with special emphasis on service induced 
constraints. Therefore we introduce a cost-function based 
approach to queueing control and provide sufficient conditions 
for its throughput optimality. The inherent complexity is tack- 
led by a randomization approach. Furthermore we show how 
particular buffers can be steered to desired operating points 
with an appropriate cost function design, which can be used 
for example to avoid buffer underflows in a streaming-traffic 
based scenario. We evaluate different cost functions for their 
suitability using numerical simulations of a large entertainment 
network. Eventually we demonstrated how the approach can 
aid the network design process, by a priori determining how 



many system resources are needed to support given service 
requirements. 
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